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USABILITY  OF  KNOWLEOGE-BASEO  SYSTEMS 


1 


The  objective  of  this  research  is  to  assess  the  value  of  different  usability 
methods  to  designing  a  system  as  complex  as  a  programming  environment  for  a 
knowledge<based  architecture  that  learns.  As  well  as  traditional  empirical 
methods  observing  real  users,  analytic  and  heuristic  methods  to  assess  the 
usability  of  complex  computer  systems  in  advance  of  running  prototypes  are 
emerging  from  human-computer  interaction  (HCI)  research.  We  will  demonstrate 
the  effectiveness  of  these  obsenrational,  analytic  and  heuristic  usability  methods 
with  respect  to 

•  The  classes  of  knowledge-based  systems  to  which  they  can  be  applied. 

•  The  types  of  usability  problems  they  can,  and  cannot  identify. 

•  The  types  of  suggestions  they  can  generate  for  the  design  of  improved 

knowledge-based  systems  and  the  relative  value  of  those  suggestions  for 

overall  usability  of  the  systems. 

•  The  expertise  of  the  evaluators  necessary  to  apply  the  methods 

•  The  effort  involved  in  applying  the  methods. 

1  Assessing  usability  methods 

Accomplishments  (July-September); 

•  We  began  work  on  an  programming  environment  for  Soar,  a  learning 
knowledge-based  production  system  language.  We  chose  to  base  our 
environment  on  the  Genie  architecture,  shown  effective  for  teaching 
procedural  programming  languages  (e.g.  Pascal,  Object-Pascal).  The 
resulting  SoarGenie  will  be  a  fleMbie  testbed  for  dec  jn  ideas  and  a 
vehicle  for  assessing  the  value  of  usability  methods.  The  Genie 
architecture  provides  a  syntax-  and  semantics-directed  editor,  templates 
for  common  coding  practices,  auxiliary  views  of  the  code  including 
browsing  and  graphical  views,  runtime  support  for  programmer  activities, 
and  hooks  to  a  multimedia  authoring  tool  for  creating  program 
documentation  and  tutorials. 

•  The  Macintosh  version  of  Soar6  was  made  operational  and  is  available 
via  anonymous  ftp.  This  version  of  Soar  will  be  the  underlying  system  for 
the  SoarGenie. 

Objectives  (October-Oecember) 

•  Bring  on  a  post-doctoral  researcher  to  apply  an  analytic  or  heuristic 
usability  method  and  assess  its  usefulness  for  designing  the  SoarGenie. 

•  Collect  empirical  usability  testing  data  for  expert  and  novice  Soar  users  in 
the  current  emacs-based  programming  environment  to  serve  as  a 
baseline  against  which  the  SoarGenie  will  be  evaluated. 

•  Continue  implementing  the  SoarGenie. 
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2  Mi^or  equipiiient  arrivals 

None. 

3  Kay  parsonnal  changsa 

_  Origir^persormei  brought  onto  the  project  in  this  initial  time  period  were 
Bonnie  E.  John  (PI)  and  Vernon  Harmon  (SoarQenie  programmer). 

4  Noteworthy  meetinga 

None. 

5  Problam  issues 

None  at  this  time. 


